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Abstract 

Over the last decade interplanetary telecommunication capabilities have been significantly 
expanded — specifically in support of the Mars exploration rover and lander missions. NASA is 
continuing to drive advances in new, high payoff optical communications technologies to enhance the 
network to Gbps performance from Mars, and the transition from technology demonstration to operational 
system is examined through a hybrid RF/optical approach. Such a system combines the best features of 
RF and optical communications considering availability and performance to realize a dual band trunk line 
operating within characteristic constraints. Disconnection due to planetary obscuration and solar 
conjunction, link delays, timing, ground terminal mission congestion and scheduling policy along with 
space and atmospheric weather disruptions all imply the need for network protocol solutions to ultimately 
manage the physical layer in a transparent manner to the end user. Delay Tolerant Networking (DTN) is 
an approach under evaluation which addresses these challenges. A multi-hop multi-path hybrid RF and 
optical test bed has been constructed to emulate the integrated deep space network and to support protocol 
and hardware refinement. Initial experimental results characterize several of these challenges and evaluate 
the effectiveness of DTN as a solution to mitigate them. 

Introduction 

Over the last decade interplanetary telecommunication capabilities have been significantly expanded, 
and specifically in support of the Mars exploration rovers Spirit (MER-A) and Opportunity (MER-B), 
Phoenix Lander and orbital missions such as Mars Express, Mars Odyssey, and Mars Reconnaissance 
Orbiter. This trend will continue with the NASA Curiosity rover, the planned MAVEN orbiter, ESA 
ExoMars rover, potential sample return (Ref. 1) and/or human exploration precursor missions (Ref. 2) 
under the Mars Next Generation Program Planning Group. Orbital relays have provided increased energy 
efficiency per bit and greater volume of science data return from resource constrained surface assets while 
concurrently supporting the onboard orbital science objectives. This relay approach will continue into the 
next decade of Mars exploration which may include joint NASA/ESA orb iters and rovers along with 
architecture developments to enable future sample return and deep space human exploration initiatives. 
Such activities will demand a communications availability and quality of service far beyond what has 
been deployed thus far in order to monitor critical events such as precision rendezvous and docking, 
distribute timing information, supplement navigational services and in general utilize the exploration 
vehicles and science spacecraft operating on or near the Mars surface to their fullest potential. 

The NASA Space Communication and Navigation (SCaN) program’s integrated network development 
plan places an emphasis on interoperability to facilitate compatibility between missions and with other space 
agencies which will influence the requirements levied on deep space relays to support multiple and evolving 
operations. A notional example of the communications architecture is shown in Figure 1, which illustrates 
the continued use of relay communications payloads on science orbiters along with dedicated 
communication relay satellites with store-and-forward and space internetworking capabilities. 
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Figure 1. — Mars relay capability (Ref. 3). 


The future Mars robotic science and human exploration precursor missions will require enhancements 
to the existing telecommunications infrastructure to include extended space internetworking services for 
each of the nodes, and integrated network management to operate the assets as essential elements of the 
SCaN service framework. Furthermore the infusion of optical communications technologies into the 
network promises a substantial enhancement in link performance with higher data rates unbounded by 
spectrum regulations. Optical communications offers secondary benefits in the form of lower transceiver 
size, mass and prime power requirements. The component technologies necessary to realize such a system 
are available now, and the overarching challenge is how to introduce the capability into a deep space 
network while minimizing spacecraft resources devoted to communications. 

These areas of advancement are investigated at the NASA Glenn Research Center (GRC) under the 
Delay/Disruption/Disconnection Tolerant Networking (DTN) and Integrated Radio and Optical 
Communications (iROC) projects. The potential capabilities and infusion methodologies of DTN and 
iROC into the operational deep space network are being examined on the hardware and software sides of 
a flight communications payload. A multi-hop multi-path hybrid RF and optical test bed has been 
constructed to emulate the integrated deep space network and to support protocol and hardware 
refinement. Initial experimental results characterize several of these challenges and evaluate the 
effectiveness of DTN as a solution to mitigate them. 


RF/Optical System Description 

One of SCaN’s primary goals is to develop a communications infrastructure that provides the highest 
data rates feasible so as to not limit the scientific capability of missions. While the exact communications 
system performance necessary to accomplish the future Mars missions is yet to be decided, it is clear that 
higher dates rates must be attained to sustain multiple synthetic aperture radar (SAR) and hyperspectral 
imaging instruments along with the demands of supporting a future human exploration initiative. 

NASA is driving advances in new, high payoff optical communications technologies to meet this 
challenge and several technology demonstration missions are scheduled over the next few years. The Jet 
Propulsion Laboratory’s (JPL) Deep Space Optical Terminal (DOT) project has consisted of a focused 
technology development effort that has matured many key technologies including pulse-position 
modulation (PPM) schemes, high-bandwidth control loops for fine beam pointing, stabilization platforms, 
precision ranging and radiation-hard leaser transmitters (Ref. 4). Much of this enabling technology has 
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been carried over and further refined from the Mars Laser Communications Demonstration (MLCD) 

(Ref. 5) project, and these advancements serve to broaden the trade space of available tools to construct 
deep space optical communication systems. The DOT effort will go a long way toward mitigating the 
perceived risks of optical communications in a deep space environment and the continual development of 
the critical enabling technologies will further extend the feasible performance of such systems to realize 
Mb/s data rates from Jupiter and Uranus (Ref. 6). NASA’s Lunar Laser Communications Demonstration 
(LLCD) is also making a similar technology push and has brought along the advancements with 
superconducting photon counting detectors and high peak-power doped-fiber transmitters (Ref. 7). 

Other critical technologies toward making successful high speed optical communications systems 
include the development of atomic clocks and oscillators capably of exceeding ~10 13 stability (Ref. 8). 
Clock distribution and synchronization between deployed assets is crucial for navigation, radiometric 
science, beam pointing control and data synchronization between network nodes. The onboard clocks 
systems must be able to periodically correct for relativistic time dilations due to gravitational redshift, and 
be robust enough to handle the ionizing radiation characteristic of the space environment. The ability to 
maintain accurate synchronization between each of the network assets is crucial towards enhancing the art 
of deep space operations and harnessing the full potential of the enabling physical layer technologies. 

The evolution of optical communications technology will enhance the SCaN network to Gbps 
performance from Mars, and the transition from technology demonstration to operational system is 
examined through a hybrid RF/optical approach (Ref. 9). The iROC project (Fig. 2) is taking this 
approach, in an effort to accelerate the infusion of optical communication technologies into the network 
timeline through realizing that both RF and optical systems must coexist on the spacecraft for some 
period of time at a minimal size, mass and power penalty to the mission. Such a system combines the best 
features of RF and optical communications considering availability and performance to realize a dual 
band trunk line operating within characteristic constraints of current infrastructure and projected 
development resources. The iROC approach takes advantage of integration strategies primarily through a 
combined RF/optical aperture, shared gimbal, integrated high bit rate digital core encoders/modulators, 
duty cycle scheduling of prime power and sharing of the supporting structural and thermal elements. 
Through the leveraging of previously mentioned technologies under the DOT and LLCD projects, iROC 
li nk budget analysis suggests feasible data rates in the hundreds of Mb/s for both the RF and optical 
connections direct to Earth (DTE) from a Mars orbiter. 

Harnessing the link capability through a networking architecture will be essential toward utilizing the 
deployed communications systems in an efficient manner to maximize overall throughput throughout a 
lifecycle that may see evolutions from robotic missions. Disconnection due to planetary obscuration and 
solar conjunction, link delays, timing, ground terminal mission congestion and scheduling policy along 
with space and atmospheric weather disruptions all imply the need for network protocol solutions to 
ultimately manage the physical layer in a transparent manner to the end user. Furthermore the data rate, 
quality of service and security needs for future deep space robotic, sample return and human exploration 
missions will be different, and the deployed integrated network solution must be general enough to adapt 
over time to address these demands. 
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Figure 2. — Dual-band deep space communications trunk line (Image courtesy APL). 
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Future communications assets will need to be implemented in compliance with Consultative Committee 
for Space Data Systems (CCSDS) standards and Interagency Operations Advisory Group (IOAG) space 
internetworking recommendations to maximize interoperability between partner agencies, and to maximize 
commonality in network and service management. Delay/Disruption/Disconnection Tolerant Networking 
(DTN) is an approach under evaluation which addresses these challenges. DTN may also enable seamless 
infusion of integrated RF/optical links into existing and future networks. Ultimately it is the role of DTN to 
make the minutia and idiosyncrasies of the integrated network appear as transparent as possible to the end 
user, while maintaining a quality of service appropriate for each traffic type. 

An examination of the integrated approach has been made for even larger distances out to 1 000 AU with 
the Realistic Interstellar Explorer (RISE) (Ref. 1 0) advanced concept. In the RISE design, the considerable 
range prohibits interactive control of the spacecraft from Earth, necessitating autonomous pointing operation 
based on celestial navigation. The optical link is used primarily for data return, and the RF system is used 
for command uplink. The novel integration of the RISE design offers a compact and power efficient 
payload, but will require higher wall-plug efficient and lightweight components to be developed. 

In the private sector side. Planetary Resources is designing spacecraft for water and precious mining of 
asteroids and their designs are based around a core integrated optical communications technology which 
would provide image capturing, navigation duties through star tracking and laser-based communication to 
Earth all through the same instrument (Ref. 11). The spirit of this integration philosophy may be traced 
back to the NASA Lunar Reconnaissance Orbiter’s Mini-RF payload which provided both SAR along 
with S and X-band communications capability from a single instrument. Moving toward the future, 
customers wanting increased functionality from smaller payloads will drive further architecture 
integrations considering reconfigurability and reusability of spacecraft elements during different 
operational states. Such an evolution will challenge the classical spacecraft architecture definitions of 
mapping a single function to a single subsystem, and will ultimately achieve more operational duty cycle 
from the elements as they contribute more towards the overall functionality of the spacecraft. 

Finally, free space hybrid RF/optical networking has been closely examined on board airborne 
platforms through the DARPA Optical RF Communications Adjunct (ORCA) program and its follow-on, 
Free-space Optical Experimental Network Experiment (FOENEX) program (Ref. 12). ORCA/FOENEX 
took advantage of layer 2 retransmission triggers to quickly recover lost data. This method has been used 
successfully to delivery data at a high 99 percent reliability with minimal impacts on latency. It is 
important to note that due to the relatively close proximity of the nodes in ORCA/FOENEX, hybrid li nk 
failover from one disrupted link to another functional one was possible through the hybrid router, but a 
scheme such as this would be difficult to implement in a deep space system due to the large link delay and 
thus feedback latency. 


Connectivity Analysis 

General concept of operation guidelines for near Earth and deep space scenarios may be found in 
Reference 13, and these may be tailored to meet specific mission criteria. Perhaps one of the most 
constrained missions would be the combined orbital science and data relay orbiter. These missions 
nominally serve a 2 year science objective, followed by a 3 year relay objective. Since the asset is placed 
in a suboptimal orbit from a telecommunications perspective (so as to facilitate the necessary imaging 
resolutions), the connectivity with Earth is very sporadic due to planetary obscuration. Furthermore, the 
orbiter may need to have a complimentary operational duty cycle between the scientific instrumentation 
and the communications system due to pointing and trajectory limitations created by gimbal constraints. 

In considering deep space scenarios, the important characteristics of large and varying range distances 
and varying geometric angles between the Sun, Earth and spacecraft drive the variable performance and 
scheduled connections of the communication li nks over a considered mission lifetime. Each connection 
would initialize with the acquisition sequence, after which data would be transferred utilizing an 
underlying synchronization format, and ending with either an orderly termination of the link or an 
unscheduled interruption that must be handled to reestablish the data transfer. 
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For the deep space hybrid RF/optical li nk to be successful it needs to be engineered as an integral part of 
a larger system. In other works, if the network is assembled in a piecewise fashion from each of its 
individual links, it will be lacking the autonomy to adapt in an efficient manner to the operational 
environment and evolving demands from the users as new missions are introduced. On the software side 
iROC is a high bit rate link, although the RF and optical links will have widely disparate rates at times. The 
conditions under which iROC nodes operate present a variety of challenges that must be overcome to be 
rendered useful. Firstly, there is disconnection due to scheduled ground station handoffs which disrupt 
connectivity, unplanned obscuration of the optical beam from atmospheric effects, logistics issues with from 
the Laser Clearing House and FAA if an uplink beacon is used, deep atmospheric fading and competition 
with other assets considering resource scheduling and management. Secondly, the round trip light times will 
at times exceed 30 min and this high latency makes it difficult to get receipts. iROC networks will need 
buffering to handle the disconnections, routing to move data across a multi-hop multi-path store and forward 
system and security to protect from the inside as well as the outside of the network. 

There have been many studies made to determine the optimal number and location of optical ground 
terminals around the globe, but to move forward with deploying a deep space communications package a 
significant performance metric must be attainable with a minimum of infrastructure. Investigations have 
been made into the utilization of a single optical site (SOS) in order to work within practical resource 
constraints, and this has proven to be a cost effective way to initially establish a capable operational 
system that can expand over time (Ref. 14). The SOS approach will place a demand on the amount of 
spacecraft memory required for buffering, but this cost is negligible compared with the construction of 
global terrestrial optical ground terminal construction. 

Several Mars architectures have been recommended, and the direction that the current system takes is 
dependent on many factors ranging from political to financial to technical. In general, the topology would 
consist of multiple surface assets talking back to multiple Deep Space Network (DSN) sites at Earth 
through multiple Mars Telecommunications Orbiters (MTO). Figure 3 is a depiction of such a system, 
with the addition of an Earth Telecommunications Orbiter (ETO). This is a useful starting point to 
conduct analysis, as it represents a multi-node multi-hop system that may be easily scaled to represent 
similar and evolving architectures. 

One unique characteristic of an integrated RF/optical payload is a co-boresighted aperture, which may 
allow simultaneous transmission on both the RF and optical channels. A concept of operations for such an 
arrangement may take advantage of the spatial proximity of the Goldstone DSN complex with the planned 
optical ground terminal. Since the RF beamwidth of a typical antenna transmitting from Mars is multiple 
times the diameter of the Earth, it can be realized that anytime the optical link is active there may be a 
simultaneous RF link established with Goldstone. This presents an opportunity for the network to operate in 
a multicast mode, which is a way of addressing multiple nodes at once. As the ground network expands 
these opportunities will increase, and the network should be flexible enough to accommodate them. 

Satellite Tool Kit (STK) may be used to determine the connectivity of a combined RF/optical Mars 
mission. Using the MRO orbit as a baseline for comparison and the SOS concept, the total daily contact 
times between the DSN sites and the optical site may vary between 10 to 20 hr depending on the time of 
year. These values represent the reductions due to elevation restrictions, scheduling and weather masking 
at the ground receivers, and in general the total daily contact times are realized from many passes lasting 



Figure 3. — Potential Mars-Earth network topology. 
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anywhere from 40 to 70 min. The considerable amount of daily contacts imply many disconnections and 
handoffs that will have to be handled at the network layer to minimize the burden on the spacecraft 
storage through intelligent retransmissions when necessary. The scheduling output from the STK 
simulations will be used to conduct the subsequent experimental emulations on the optical test bed. 


Experimental Setup 

Previous work at JPL has reported on an initial experiment characterizing the performance of a DTN 
enabled point-to-point space optical link, and through this it was learned that excellent system 
performance can be achieved on a disconnected link by using link-state information to burst high rates of 
data upon reacquisition (Ref. 15). The experimental setup picks up where the point-to-point link left off. 
DTN adds the capability needed to think about iROC in bigger terms than a simple point-to-point link, 
and this is realized through a multi-node multi-hop test bed to serve as a research platform for new DTN 
software components and provide honest network characterization for present and future missions. 

The initial test bed consists of five network nodes connected by six free space optical links. One end 
of the test bed reaches out to the Mars surface robot to receive data to be relayed through the network and 
channel emulator, and the other side of the test bed delivers the routed data to the Mission Operations 
Center (MOC) back at Earth (Fig. 4). The simulation results from STK are used to drive the network, so 
the experiments are conducted in a manner that emulates realistic mission contacts and disconnections. 

Each of the computer nodes in the network are connected through a bidirectional optical Ethernet 
convertor, which is launched into a free space optic system on the optics bench (Fig. 5). 

The test bed is modeled after a scenario in which a Mars orbiter sends data to a MOC. The orbiter 
sends data direct-to-earth (DTE) and has contact times scheduled with three different deep space network 
(DSN) sites. The DSN sites are also networked to the MOC. This is a multi-hop multi-path network. 

To generate realistic connectivity times and link delays, the scenario was laid out in the STK software 
package. A Mars orbiter was placed along with three DSN sites (Canberra, Goldstone, and Madrid). The 
data, summarized in Azimuth, Elevation, and Range (AER) reports from which the DTN configurations 
were distilled. The selected time period was over the month of June in 2012. 



Figure 4. — Multi-node multi-hop free space optical DTN test bed. 
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Figure 5. — Optical ethemet convertors and free space optics. 


There are six computers in total. Five of the computers are the DTN nodes (The orbiter, the MOC, 
and the three DSN sites). The sixth computer is a channel emulator that reproduces the Mars to Earth li nk 
characteristics — notably the delay. During the chosen test period, Mars was on average 1 0 min away by 
light (20 min round trip time). There are multiple implementations of DTN, and ION version 3.0.1 was 
selected. ION, developed by JPL and NASA, computes routes based on known contact times. This is 
valid in the network described, as the assets' connectivity are all determined by orbital mechanics (Mars 
to Earth) or are constant (DSN sites to the MOC). The channel emulator is an open software package 
developed at the NASA Glenn Research Center for earlier projects. 

The CPUs in the computers are all first generation Pentium 4s, and each computer has 512 MB of 
RAM. As satellites are limited platforms by terrestrial standards, it would not be fair to have modem 
machines. Each computer runs Ubuntu 11.10. Due to the 1 0 min link delay, the Address Resolution 
Protocol (ARP) had to be set statically. Additionally, significant IPv6 traffic was observed over the li nks 
with 10 min delays and therefore IPv6 was disabled on all machines. The primary unit of data in DTN, 
the bundle, has a timeout based on seconds. Furthermore, connectivity is time based, so all computers had 
to be synchronized. Clock drift was observed even in the lab environment, so an Ethemet backchannel 
was present to all nodes so that one could act as a Network Time Protocol (NTP) server to achieve 
synchronization. 


Preliminary Results 

The basic actions of the generic DTN are to store, carry, and forward data. All networks forward data 
towards the recipient or recipients. The storage of large amounts of data for prolonged amounts of time 
and the potential mobility required to carry data to the next forwarding agent (also known as a hop) set 
DTN apart from traditional networking concepts. Each of these simple words runs deeply with both 
known and u nk nown implications that are being uncovered and resolved with modem DTN research. 

Storing data enables a disconnected node to wait for a contact period with a neighbor. During 
disconnection it is likely that additional data will be added to the buffer. This only partially determines 
local buffer requirements; indeed, if a node has to store its own data and a neighbor’s, then there will be 
contention for limited resources. The notion of policies to ensure gracefulness is an ongoing area of 
research. Another capability added by carrying data is reliability — if the data is held even after 
transmission, then retransmissions are straightforward. Perhaps a more subtle difficulty overcome with 
storage is that of link asymmetry. If a node receives data quicker than it can send it, the data must be 
buffered or lost. DTNs handle this problem automatically; however this requires careful planning for 
storage requirements. 
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Carrying data allows nodes to either wait for a neighbor to arrive or to move and go to a neighbor. As 
a node moves, perhaps in orbit, it may receive data from neighbors. As described, this might complicate 
matters from a storage standpoint as the policies local to the orbiting node cannot be synchronized with 
nodes passed. As connectivity might not be possible (no line of sight) or link delays get too large, 
feedback loops are not possible. Another difficulty encountered with the carrying of data is that 
addressing is not well-defined in DTN. Addressing in a network tells where a node is so that when 
forwarding data a route can be chosen so that with each transmission the data moves closer to its 
destination. Without an addressing structure this information must be determined by other means. In our 
case, this is possible as all connection periods can be calculated. This is not possible with all nodes in a 
more general network. 

When testing ION in our network, the ultimate goal is to get as much data from Mars to the Earth as 
possible. This implies reliability on ION’s part as well as efficient li nk utilization. Every time a bundle is 
sourced from the Mars orbiter node, new information has to be added to the bundle queue within ION. 
This queue will be at its largest when many outgoing bundles are added during periods of complete 
disconnection, during which time the behavior of ION will come under particular scrutiny. 

Part of testing protocols is understanding their breaking points, and we uncovered one such point when 
sending bundles larger than 64 MB in our configuration. Therefore, the largest bundles sent were 64 MB. 
We chose to send one 64 MB bundle and then one random bundle (10 k, 100 k, 1 M, or 10 M) and repeat 
every three seconds. The original intention was to run two tests — one with custody transfer and one 
without. Custody transfer is the aforementioned mechanism by which retransmission may occur if 
necessary. On a spacecraft, all processes are granted a fixed amount of memory. What was originally 
computed to be sufficient for ION turned out to be incorrect, and after a while new bundles were 
immediately dropped by the Mars node. 

The memory was thus increased as far as ION would allow without crashing. It turned out that the 
additional memory was also insufficient, but during a different period. The first test, with the smaller 
memory pool, ran out of memory during a contact period. The first test was rerun with more memory which 
ran out of memory during a period of total disconnection. The test with custody transfer was also run with 
the larger amount of memory, which ran out of memory at the same point as the first repeated test. 

This result actually makes a lot of sense. If larger bundles could be sourced, then fewer entries would 
be added to the queue. This would prevent overloading the protocol and present less of a burden to the 
processor and memory, but more of a burden to the non-volatile storage (hard drive or solid state drive). 
The periods of disconnection lasted on the order of hours, and memory ran out after 2700 additional 
bundles were sourced which is fair. This problem was amplified by the fact that the transmission from 
Mars to the DSN sites ran much slower than line rate and therefore the three second delay between 
transmissions was insufficient. It was found that with custody transfer nearly half of the received bundles 
were 64 MB, which reflects the sender’s situation. Without custody transfer more bundles were received 
but less than 35 percent of the received bundles were 64 MB. 

To avoid having thousands of bundles in the queue there are several possible mechanisms. One is to 
extend the protocol to send larger bundles, which in simpler circumstances it already can. Additionally, it 
may be useful to aggregate smaller bundles into larger bundles. For example, during a period of 
disconnection, ION might try to assemble bundles that are at least 512 MB. This would resemble the way 
Amazon ships books to consumers — not one book per box. Note that this approach is not entirely custody 
transfer friendly. Reliability in the form of retransmissions would become more expensive. It may be that 
not every piece of an aggregate bundle needs to be retransmitted, but to send pieces the custody signal 
would have to be at least somewhat content aware to avoid undue link congestion and at any rate the 
entire aggregate would have to be stored on the space craft. This and other considerations on the order of 
fragmentation merit further study. 
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Conclusions and Future Work 


The initial test bed to support hybrid RF/optical deep space investigations for iROC and DTN has 
been successfully constructed and operated. Initial implications over memory allocation and custody 
transfer have been highlighted in the mission scenario, and there now exists a capability to research 
further into these and subsequent issues. Over time it is planned to expand the test bed to conduct 
complete network experiments featuring RF, optical and wired (ground) links, which will allow further 
research into multicast and dynamic routing capabilities. More relay assets will be added, including the 
UHF links typical of resource constrained surface instruments. Physical layer channel emulations of 
atmospheric fades, background noise, unscheduled disconnections and pointing losses are also in the 
planning stages to evaluate the robustness of the network. 

Further DTN testing will explore security issues and policy management, along with developing 
requirements for on board memory, software footprint and power requirements. Preparations are currently 
underway to expand the link range of the test bed by replacing the Ethernet to fiber convertors with dedicated 
1550 nm lasers and electro optic modulators, which are launched into telescopes to place the detectors as far 
away as 1500 m (Fig. 6). This new arrangement will allow a portion of the test bed to be operated in actual 
weather conditions, thus adding to the unpredictable and honest evaluation of the network. 



Figure 6. — Launching fiber-coupled laser into telescopes. 
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